Web Survey Bibliography
It is common survey practice to convert a series of yes/no (forced-choice) items in telephone surveys to check-all-that-apply items in web and mail surveys. However, relatively little is known about how these different question formats may influence answers. This paper reports results from two web experiments and a comparison paper experiment in which nine different questions, varying in substantive topic and type (opinion-based and fact/behavior-based) were asked in 16 experimental comparisons of the check-all and forced-choice formats. Our purpose was to determine whether this change in question format influenced the number of response options marked affirmatively within each question and why any differences might occur. Results revealed that in every instance respondents marked significantly more items in the forced-choice format than in the check-all format. Given these results, detailed analyses of response patterns within questions, answering time, and alternative wording structures of questions were undertaken to examine which of three theories (satisficing, depth of processing, and acquiescence) best accounted for the response differences across question formats. These analyses indicated that the forced-choice format appears to invoke deeper processing and to eliminate satisficing behavior that occurs among some respondents to the check-all format, but that acquiescence does not seem to be an issue in the forcedchoice format. Thus, it appears that the use of the forced-choice question format is a desirable alternative to the use of the check-all question format for multiple answer questions. In addition, the findings reported here give ample reason to be concerned about the current practice of automatically converting items from the forced-choice format to the check-all format or vise versa when switching between telephone and paper or web surveys.
Conference program
Web survey bibliography - Smyth, J. D. (23)
- The Effect of CATI Questions, Respondents, and Interviewers on Response Time; 2016; Olson, K.; Smyth, J. D.
- Identifying predictors of survey mode preference; 2015; Millar, M. M.; Olson, K.; Smyth, J. D.
- Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 4th Edition; 2014; Dillman, D. A., Smyth, J. D., Christian, L. M.
- The Effect of Answering in a Preferred Versus a Non-Preferred Survey Mode on Measurement; 2014; Smyth, J. D., Olson, K., Kasabian, A.
- Assessing Within-Household Selection Methods in Household Mail Surveys; 2014; Olson, K., Stange, M., Smyth, J. D.
- Accuracy of Within-household Selection in Web and Mail Surveys of the General Population.; 2014; Olson, K., Smyth, J. D.
- Using Eye Tracking to Examine the Visual Design of Web Surveys; 2014; Zhou, Q., Ricci, K., Olson, K., Smyth, J. D.
- Are You Seeing What I am Seeing? Exploring Response Option Visual Design Effects With Eye-Tracking; 2013; Libman, A., Smyth, J. D., Olson, K.
- The effects of item saliency and question design on measurement error in a self-administered survey; 2012; Stern, M. J., D., Mendez, J. D.Smyth, J. D.
- Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates?; 2012; Olson, K., Smyth, J. D., Wood, H.
- Turn that Frown Upside-Down: The Effects of Smiley Faces as Symbolic Language in Self-administered Surveys...; 2012; Libman, A., Smyth, J. D.
- Literacy and Data Quality in Self-Administered Surveys; 2011; Smyth, J. D., Olson, K.
- Medium Node: NSF Census Research Network; 2011; McCutcheon, A. L., Belli, R. F., Olson, K., Smyth, J. D., Soh, L.-K.
- Using the Internet to Survey Small Towns and Communities: Limitations and Possibilities in the Early...; 2010; Smyth, J. D., Dillman, D. A., Christian, L. M., O'Neill, A. C.
- Response format effects on measurement of employment; 2009; Thomas, R. K., Dillman, D. A., Smyth, J. D.
- Open-Ended Questions in Web Surveys: Can Increasing the Size of Answer Boxes and Providing Extra Verbal...; 2009; Smyth, J. D., Dillman, D. A., Christian, L. M., McBride, M.
- ...; 2008; Dillman, D. A., Smyth, J. D., Christian, L. M., O'Neill, A. C.
- Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method ; 2008; Dillman, D. A., Christian, L. M., Smyth, J. D.
- Combining Mail and Internet Methods to Conduct Household Surveys of the General Public: A New Methodology...; 2008; Dillman, D. A., Smyth, J. D., Christian, L. M., Oneill, A.
- Context effects in Internet Surveys: New issues and evidence; 2007; Smyth, J. D., Dillman, D. A., Christian, L. M.
- Is it Possible to Obtain Equivalent Answers to Scalar Questions in Web and Telephone Surveys?; 2006; Christian, L. M., Dillman, D. A., Smyth, J. D.
- Open-Ended Questions in Web and Telephone Surveys; 2006; Smyth, J. D., Dillman, D. A., Christian, L. M., McBride, M.
- Comparing Check-All and Forced-Choice Question Formats in Web Surveys: The Role of Satisficing, Depth...; 2005; Smyth, J. D., Dillman, D. A., Christian, L. M., Stern, M. J.